Method and apparatus for detecting scene changes in video using a histogram of frame differences

ABSTRACT

The present invention is directed to method of detecting a scene change in video. The method includes a difference being calculated between consecutive frames of the video producing a plurality of difference values. A histogram is taken of the difference values. A scene change is detected if a majority of the difference values in the histogram exceed a predetermined value.

BACKGROUND OF THE INVENTION

The present invention relates generally to video processing, and moreparticularly to, detecting scene changes in video using a histogram offrame differences.

Scene change detection is used to detect a significant change in contentbetween consecutive frames in a video sequence. For example, significantchanges in content occur when a program switches to a commercial oranother program, or switches to a different scene in the same program.

Accurate scene change detection may be used to improve the performanceof different video processing algorithms. For example, it may be used invideo enhancement algorithms to adjust certain parameters for differentscene content. Scene change detection may also be used in videocompression algorithms. For example, it may be used to determine when anew group of pictures has to be started or to anticipate bit ratechanges.

A number of scene change detection techniques already exist. Onetechnique take histograms of individual frames and then performs acomparison between the histograms of consecutive frames in order todetect a scene change. This technique works well in some situations.However, problems arise when the consecutive frames have similar pixelvalues in different positions. In this situation, the histograms ofthese two frames would be quite similar and thus would prevent a scenechange from being detected.

Another technique uses coding information in order to detect a scenechange. For example, this technique looks for an increase in aparticular type of block in B-frames and P-frames in order to detect ascene change. This technique works well for B-frames and P-frames,however, it cannot be used for I-frames.

SUMMARY OF THE INVENTION

The present invention is directed to method of detecting a scene changein video. In one example, the method includes a difference beingcalculated between consecutive frames of the video producing a pluralityof difference values. A histogram is taken of the difference values. Ascene change is detected if a majority of the difference values in thehistogram exceed a predetermined value.

In another example, the method includes determining if a frame of videois an I-frame. A scene change is detected based on a histogram of framedifferences if the frame is an I-frame. Alternatively, a scene change isdetected based on coding information if the frame is not an I-frame.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings were like reference numbers representcorresponding parts throughout:

FIG. 1 is a flow diagram of one example of detecting a scene changeaccording to the present invention;

FIG. 2 is a graph illustrating an example of the characteristic value Q;

FIG. 3 is a block diagram of one example of a video device according tothe present invention; and

FIG. 4 is a flow diagram of another example of detecting a scene changeaccording to the present invention.

DETAILED DESCRIPTION

The present invention is directed to a scene detection technique thatmay be used in any situation. According to the present invention, thistechnique uses the histogram of the differences between two frames inorder to detect a scene change. Further, a strategy is also proposed toreduce the computational complexity of scene change detection for MPEGencoded video by exploiting coding information from the compressedbit-streams.

One example of a method for detecting a scene change in video accordingto the present invention is shown in FIG. 1. For the purpose of thisdiscussion, it is presumed that the frames of video being processed bythe method of FIG. 1 have already been decoded. In step 2, a differencebetween consecutive frames of the video is calculated. This isaccomplished by calculating the difference between corresponding pixelvalues in each of the frames, which is defined as follows:D({right arrow over (x)})=|(f({right arrow over (x)},n)−f({right arrowover (x)},n−1))|  (1)where ${\overset{->}{x} = \begin{pmatrix}x \\y\end{pmatrix}},$f({right arrow over (x)},n) is the pixel value of the frame n at thelocation {right arrow over (x)}, f({right arrow over (x)},n−1) is thepixel value of the previous frame n−1 at the same locations. As a resultof step 2, a plurality of difference values are produced that are themagnitudes of the difference between corresponding pixel values in theframes.

According to the present invention, the difference calculated in step 2may be for all of the pixel values in each of the frames. Alternatively,in order to reduce the number of computations required, the differencemay be calculated for a selected portion of the frames. For example, thedifference may be calculated for pixel values only in the center of theframes.

In step 4, a histogram of the difference values is taken. A histogram isa well known function for plotting the distribution of a particularvariable. In this case, the distribution is defined as His(i):, which isthe number of the difference values that equal a particular value i. Aspreviously described, the difference values represent the magnitude ofthe difference between corresponding pixel values in the frames.

In step 6, a scene change is detected if a majority of the differencevalues in the histogram exceed a predetermined value. For the purpose ofthe discussion, a majority is defined as any number over fifty (50)percent. However, depending on the predetermined value selected, themajority of the difference values required to detect a scene change maypreferably be greater than fifty (50) percent. For example, for a valueof ten (10), it is preferable that sixty-three (63) percent of thedifference values in the histogram exceed this value.

In order to perform step 6, it may be preferable to find acharacteristic value Q, which satisfies the following condition:$\begin{matrix}{\frac{\sum\limits_{i = 0}^{Q}{{His}(i)}}{\sum\limits_{i = 0}^{2^{n} - 1}{{His}(i)}} > {T1}} & (2)\end{matrix}$where His(i) is the histogram value at i and n is the bit precision usedby each frame of the video. T1 is a threshold, which in one example maybe a value of 0.368. In this example, the characteristic value Qrepresents a particular difference value in the histogram where 36.8percent of the difference values have a smaller value. Therefore, Qdefines a predetermined range of the different values in the histogram.

An example of the characteristic value Q is shown in FIG. 2. As can beseen, Q corresponds to a particular difference value defining apredetermined range. In FIG. 2, the probability is defined as:$\begin{matrix}{{{Prob}(j)} = \frac{\sum\limits_{i = 0}^{j}{{His}(i)}}{\sum\limits_{i = 0}^{{2n} - 1}{{His}(i)}}} & (3)\end{matrix}$As an example in FIG. 2, prob(M)=1 and P=prob(Q)>0.368.

After finding the characteristic value Q, a scene change is detected ifthe following condition is met: Q>T2. Where T2 is a threshold value. Inone example, the threshold value T2 may have a value of ten (10).

One example of a video device according to the present invention isshown in FIG. 3. By way of example, the device may represent atelevision, a set-top box, a desktop, laptop or palmtop computer, apersonal digital assistant (PDA), a video/image storage device such as adigital video disc (DVD) player, a digital video recorder (DVR), a TiVOdevice, etc., as well as portions or combinations of these and otherdevices.

As can be seen, the device includes a decoder 8 and a scene changedetection unit 10. In one embodiment, the device of FIG. 3 may beimplemented by a computer readable code that is stored on a memorymedium such as a CD-ROM or floppy disk that can be executed by aprocessor. In other embodiments, hardware circuitry may be used in placeof, or in combination with, software instructions to implement thedevice of FIG. 3.

During operation, the decoder will decode the incoming bit stream inorder to produce decoded video frames. According to the presentinvention, the decoder may be a MPEG-1 decoder, MPEG-2 decoder, MPEG-4decoder, H.26x decoder or another similar type. The scene changedetection unit 10 will then look at the decoded video in order todetermine if a scene change has occurred. If a scene change is detectedin one of the video frames, the scene detection unit 10 produces a flagidentifying which frame has a scene change.

According to the present invention, the scene change detection unit 10may implement the method of FIG. 1 in order detect a scene change. Thus,the scene change detection unit 10 will perform the method steps of FIG.1 for all of the decoded video frames. However, utilizing the method ofFIG. 1 for all of the frames may not be efficient since it requiresalways storing the previous frame in order to calculate the difference.

In view of the above, it may be desirable in some situations toimplement a strategy to reduce the computational complexity and memoryrequirement for the scene change detection unit 10. In MPEG bitstreams,there are different types of frames such as an Intra-coded (I) frame,Predicted (P) frame and Bi-directional (B) frame. Depending on the frametype, it would be desirable to use a different technique to detect ascene change. For example, for P and B-frames coding information may beused, while for 1 frames the method of FIG. 1 may be utilized.Therefore, by using this combined strategy, the computational complexityas well as the memory requirements is significantly reduced since thefrequency of I-frames is much lower than P and B-frames in typical MPEGbit-streams.

A method of detecting a scene change implementing the above-describedstrategy is shown in FIG. 4. In step 12, it is determined whether aframe of the decoded video is an I-frame. This may be accomplished bythe coding information provided by the decoder 8 in FIG. 3. Suchstandards as MPEG-1, MPEG-2, MPEG-4, and H.26x all require compliantbit-streams to include header fields identifying the type of framesincluded. Thus, during operation, as the decoder 8 decodes the incomingbit-stream, it will also look at the header field in order to determinethe type of frames in the decoded video. This information will beincluded in the coding information sent to the scene change detectionunit 10 during operation.

In step 12, if it is determined that a frame is an I-frame, then a scenechange is detected for that particular frame based on a histogram offrame differences in step 14. Therefore, the scene change detection unit10 will perform the method of FIG. 1 for that particular frame.

In step 12, if it is determined that a frame is not an I-frame, then ascene change is detected for that particular frame based on codinginformation in step 16. During operation, the decoder 8 will providecoding information to the scene detection unit 10. As previouslydescribed, the coding information includes information about the type offrames in the decoded video. The coding information will also includeinformation about the type of macroblocks that make up a particularframe of video.

For example, if a scene change occurs in a P-frame, the number ofIntra-coded macroblocks increases. Therefore, a scene change is detectedbased on the following criterion:${Scene\_ Change} = \{ {{\begin{matrix}1 & {{{if}\quad{C1}} > {T3}} \\0 & {else}\end{matrix}{C1}} = \frac{Num\_ Intra}{Num\_ MB}} $

-   -   where Num_Intra is the number of the Intra-coded macroblocks,        Num_MB is the total number of the macroblocks and T3 is a        threshold value. According to this criterion, the scene change        detection unit 10 will determine the number of intra-coded        macroblocks included in a P-frame. The scene change detection        unit 10 will then detect a scene change if the number of        Intra-coded macroblocks is greater than a predetermined        threshold, which may be twenty (20) percent or higher.

For example, if a scene change occurs in a B-frame, the number ofbackward-predicted macroblocks increases. Therefore, a scene change isdetected based on the following criterion:${Scene\_ Change} = \{ {{\begin{matrix}1 & {{{if}\quad{C2}} > {T4}} \\0 & {else}\end{matrix}{C2}} = \frac{Num\_ Back}{Num\_ MB}} $

-   -   where Num_Back is the number of the backward-predicted        macroblocks, Num_MB is the total number of the macroblocks and        T4 is a threshold value. According to this criterion, the scene        change detection unit 10 will determine the number of        backward-predicted macroblocks included in a B-frame. The scene        change detection unit 10 will then detect a scene change if the        number of backward-predicted macro blocks is greater than a        predetermined threshold, which may be seventy (70) percent or        higher.

The video device will include other units (not shown) coupled to theoutputs of the decoder 8 and scene change detection unit 10. These otherunits will utilize the flag produced by the scene change detection unit10 in order to further process the video frames from the decoder 8. Forexample, a video processing algorithm using a motion-compensated filtermay benefit from the scene change detection. During operation, themotion-compensated filtering must be turned off when a scene changeoccurs otherwise wrong results may result. The video processingalgorithm will use the scene change flag produced by the scene changedetection unit 10 in order to reset the motion compensated filteringoperation.

While the present invention has been described above in terms ofspecific examples, it is to be understood that the invention is notintended to be confined or limited to the examples disclosed herein.Therefore, the present invention is intended to cover various structuresand modifications thereof included within the spirit and scope of theappended claims.

1-6. (canceled)
 7. A video device, comprising: a decoder for decoding anincoming bit-stream to produce frames of video; and a scene changedetection unit for calculating a difference between consecutive framesof the video producing a plurality of difference values and detecting ascene change if a majority of the difference values exceed apredetermined value.
 8. The device of claim 7, wherein the scene changedetection unit further performs taking a histogram of the differencevalues.
 9. A method of detecting a scene change in video, the methodcomprising the steps of: determining if a frame of video is an I-frame;detecting a scene change based on a histogram of frame differences ifthe frame is an I-frame; and detecting a scene change based on codinginformation if the frame is not an I-frame.
 10. The method of claim 9,wherein the detecting a scene change based on a histogram of framedifferences includes: calculating a difference between consecutiveframes of the video producing a plurality of difference values; taking ahistogram of the difference values; and detecting a scene change if amajority of the difference values in the histogram exceed apredetermined value.
 11. The method of claim 9, wherein the detecting ascene change based on coding information includes: determining a numberof Intra-coded blocks in a P-frame; detecting a scene change if thenumber of Intra-coded blocks exceeds a predetermined number.
 12. Themethod of claim 9, wherein the detecting a scene change based on codinginformation includes: determining a number of backward-predicted blocksin a B-frame; detecting a scene change if the number ofbackward-predicted blocks exceeds a predetermined number.
 13. A memorymedium including code for detecting a scene change in video, the codecomprising: a code for determining if a frame of video is an I-frame; acode for detecting a scene change based on a histogram of framedifferences if the frame is an I-frame; and a code for detecting a scenechange based on coding information if the frame is not an I-frame.
 14. Avideo device, comprising: a decoder for decoding an incoming bit-streamto produce frames of video; and a scene change detection unit fordetermining if a frame of video is an I-frame, detecting a scene changebased on a histogram of frame differences if the frame is an I-frame anddetecting a scene change based on coding information if the frame is notan I-frame.